perf: optimize security-guard token usage by lpcox · Pull Request #1648 · github/gh-aw-firewall

lpcox · 2026-04-03T21:22:00Z

Summary

Optimizes the security-guard workflow to reduce token consumption based on analysis from the token optimization advisor (#1647).

Changes

1. Restrict GitHub toolsets (Rec #1)

Changed toolsets: [default] → toolsets: [pull_requests, repos]
The default toolset loads 52 GitHub MCP tools; security-guard only uses ~7 (PR diff, file contents, search)
pull_requests + repos provide all needed tools while reducing tool schema overhead at the MCP gateway level

2. Pre-compute PR diff (Rec #3)

Added steps: block that pre-fetches PR changed files via gh api before the agent starts
Injects diff data directly into the prompt body via ${{ steps.pr-diff.outputs.PR_FILES }}
Eliminates the agent's initial get_pull_request_diff + get_pull_request_files tool calls (~2 round trips saved)
Capped at 8KB to prevent prompt bloat on large PRs

3. Add max-turns limit (Rec #4)

Added max-turns: 15 to prevent runaway token consumption
Security review is a focused task — 15 turns is generous for reading a diff and posting a comment

4. Explicit network restriction

Added network: allowed: [github] to explicitly restrict egress to GitHub domains only
Security-guard doesn't need npm, pypi, playwright, or other network groups

Expected Impact

Based on token optimization advisor analysis:

~38% token reduction from toolset restriction (fewer tool schemas loaded)
~2 fewer tool call round trips from pre-computed diff
Hard cap on runaway usage via max-turns

Closes #1647

- Restrict GitHub toolsets from [default] (52 tools) to [pull_requests, repos] (only tools actually used) - Add pre-compute step to fetch PR diff before agent starts, reducing tool calls needed for initial PR analysis - Add max-turns: 15 to prevent runaway token consumption - Add explicit network: allowed: [github] to restrict egress - Update prompt to reference pre-fetched diff data Closes #1647 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-04-03T21:23:00Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	86.11%	86.21%	📈 +0.10%
Statements	85.98%	86.08%	📈 +0.10%
Functions	87.41%	87.41%	➡️ +0.00%
Branches	78.50%	78.56%	📈 +0.06%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/docker-manager.ts`	86.4% → 86.8% (+0.39%)	85.9% → 86.3% (+0.38%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

Updates the Security Guard agentic workflow configuration to reduce Claude token usage by narrowing the GitHub MCP tool surface area, limiting agent turns, and attempting to pre-inject PR diff context into the prompt.

Changes:

Restrict GitHub MCP toolsets to pull_requests + repos and add an engine max-turns limit.
Add a workflow step intended to pre-fetch PR changed-file patches and include them in the prompt.
Re-compile the generated lock workflow with a newer gh-aw compiler/action version.

Show a summary per file

File	Description
.github/workflows/security-guard.md	Adjusts frontmatter (toolsets, max-turns, network) and adds a step + prompt section intended to inject pre-fetched PR diffs.
.github/workflows/security-guard.lock.yml	Re-generated compiled workflow reflecting the updated frontmatter and newer gh-aw tooling, including toolset restriction and max-turns.
.github/aw/actions-lock.json	Adds the pinned `github/gh-aw-actions/setup@v0.65.5` entry used by the compiled workflow.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (4)

.github/workflows/security-guard.md:81

steps.pr-diff.outputs.PR_FILES is referenced in the prompt body, but the steps: block runs as GitHub Actions steps and does not execute during prompt rendering unless it runs in the same job before the prompt is generated. With the current workflow compilation, the prompt is created in the activation job while pr-diff runs later in the agent job, so this placeholder will not be substituted with the diff.

Consider moving the PR-diff fetch into the activation job (pre-prompt) or switching to a mechanism that injects the diff into the prompt via activation outputs/artifacts.

## Changed Files (Pre-fetched)

The following PR diff has been pre-computed. Focus your security analysis on these changes:

${{ steps.pr-diff.outputs.PR_FILES }}

.github/workflows/security-guard.lock.yml:316

This pr-diff step has the same output-capture issue as the .md source: the gh api ... | head -c 8000 output is written to stdout rather than to $GITHUB_OUTPUT, so PR_FILES will be empty.

Also, PR_NUMBER is sourced from github.event.pull_request.number without guarding for workflow_dispatch (where it will be empty), which will cause the gh api call to fail on manual runs. Consider adding an if: to run only when a PR number is present, or derive the PR number from inputs.

      - env:
          GH_REPO: ${{ github.repository }}
          GH_TOKEN: ${{ github.token }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
        id: pr-diff
        name: Fetch PR changed files
        run: "echo \"PR_FILES<<EOF\" >> $GITHUB_OUTPUT\ngh api \"repos/${GH_REPO}/pulls/${PR_NUMBER}/files\" \\\n  --paginate --jq '.[] | \"### \\(.filename) (+\\(.additions)/-\\(.deletions))\\n\\(.patch // \"(binary)\")\\n\"' \\\n  | head -c 8000\necho \"EOF\" >> $GITHUB_OUTPUT\n"

.github/workflows/security-guard.lock.yml:1067

Same concern as earlier in the workflow: installing @anthropic-ai/claude-code@latest in the detection job makes runs non-reproducible and can introduce unexpected breakages.

Pin to a specific version (or a controlled variable) for deterministic behavior.

          sudo chmod +x /usr/local/bin/awf
      - name: Install Claude Code CLI
        run: npm install -g @anthropic-ai/claude-code@latest
      - name: Execute Claude Code CLI

.github/workflows/security-guard.md:37

This step will fail (or fetch the wrong URL) on workflow_dispatch runs because PR_NUMBER is set only from github.event.pull_request.number, which is empty outside PR events. Add an if: guard so it only runs when a PR number exists, or accept a PR number as an explicit workflow_dispatch input and use that here.

      GH_TOKEN: ${{ github.token }}
      PR_NUMBER: ${{ github.event.pull_request.number }}
      GH_REPO: ${{ github.repository }}

Files reviewed: 3/3 changed files
Comments generated: 5

Copilot · 2026-04-03T21:26:54Z

.github/workflows/security-guard.md

+      echo "PR_FILES<<EOF" >> $GITHUB_OUTPUT
+      gh api "repos/${GH_REPO}/pulls/${PR_NUMBER}/files" \
+        --paginate --jq '.[] | "### \(.filename) (+\(.additions)/-\(.deletions))\n\(.patch // "(binary)")\n"' \
+        | head -c 8000
+      echo "EOF" >> $GITHUB_OUTPUT


The step output PR_FILES is not being populated: you write the PR_FILES<<EOF/EOF markers to $GITHUB_OUTPUT, but the gh api ... | head -c 8000 output is sent to stdout rather than appended to $GITHUB_OUTPUT. As a result, steps.pr-diff.outputs.PR_FILES will be empty.

Wrap the delimiter echoes and the gh api command in a single block that appends all lines to $GITHUB_OUTPUT (or redirect the gh api pipeline output into $GITHUB_OUTPUT between the markers).

This issue also appears in the following locations of the same file:

line 35

line 75

Suggested change

echo "PR_FILES<<EOF" >> $GITHUB_OUTPUT

gh api "repos/${GH_REPO}/pulls/${PR_NUMBER}/files" \

--paginate --jq '.[] | "### \(.filename) (+\(.additions)/-\(.deletions))\n\(.patch // "(binary)")\n"' \

| head -c 8000

echo "EOF" >> $GITHUB_OUTPUT

{

echo "PR_FILES<<EOF"

gh api "repos/${GH_REPO}/pulls/${PR_NUMBER}/files" \

--paginate --jq '.[] | "### \(.filename) (+\(.additions)/-\(.deletions))\n\(.patch // "(binary)")\n"' \

| head -c 8000

echo "EOF"

} >> "$GITHUB_OUTPUT"

Copilot · 2026-04-03T21:26:54Z

.github/workflows/security-guard.md

+    toolsets: [pull_requests, repos]
+network:
+  allowed:
+    - github


network.allowed is set to only github, but this workflow uses the Claude engine (and the generated lock file currently still allows many non-GitHub domains). If the intent is to truly restrict egress, please confirm which network groups are required for Claude/API-proxy operation and include them explicitly (e.g., the same baseline groups used by other Claude workflows), otherwise a future recompile could inadvertently block the model call path.

Suggested change

- github

- github

- api-proxy

Copilot · 2026-04-03T21:26:54Z

.github/workflows/security-guard.lock.yml

      - name: Create prompt with built-in context
        env:
          GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
-          GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }}
+          GH_AW_SAFE_OUTPUTS: ${{ runner.temp }}/gh-aw/safeoutputs/outputs.jsonl
+          GH_AW_EXPR_BAA3A6C6: ${{ steps.pr-diff.outputs.PR_FILES }}
          GH_AW_GITHUB_ACTOR: ${{ github.actor }}
          GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }}


In the activation job, GH_AW_EXPR_BAA3A6C6 is set from ${{ steps.pr-diff.outputs.PR_FILES }}, but there is no pr-diff step in this job. This will evaluate to empty (or error depending on expression handling), so the prompt won’t receive the pre-fetched diff.

Add the Fetch PR changed files step to the activation job before prompt creation, or stop referencing steps.pr-diff from activation and source the content from an activation output/artifact instead.

This issue also appears on line 309 of the same file.

Copilot · 2026-04-03T21:26:54Z

.github/workflows/security-guard.lock.yml

  activation:
-    if: (github.event_name != 'pull_request' || (github.event.pull_request.head.repo.id == github.repository_id && github.event.sender.type != 'Bot'))
+    if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.id == github.repository_id
    runs-on: ubuntu-slim


The activation gate changed from excluding bot actors to allowing them (github.event.sender.type != 'Bot' was removed). This is a behavioral change not mentioned in the PR description and may increase workflow runs on automated PRs.

If this is intentional, please call it out in the PR description; otherwise, restore the bot exclusion to preserve prior behavior.

Copilot · 2026-04-03T21:26:55Z

.github/workflows/security-guard.lock.yml

          sudo chmod +x /usr/local/bin/awf
      - name: Install Claude Code CLI
-        run: npm install -g --silent @anthropic-ai/claude-code@2.1.47
+        run: npm install -g @anthropic-ai/claude-code@latest


npm install -g @anthropic-ai/claude-code@latest makes the workflow non-deterministic and can break unexpectedly when a new Claude Code release is published.

Prefer pinning to a known-good version (or using a repo/org variable to control the version) so runs are reproducible.

This issue also appears on line 1064 of the same file.

Suggested change

run: npm install -g @anthropic-ai/claude-code@latest

env:

CLAUDE_CODE_VERSION: ${{ vars.CLAUDE_CODE_VERSION }}

run: |

if [ -z "${CLAUDE_CODE_VERSION}" ]; then

echo "CLAUDE_CODE_VERSION repo/org variable must be set to a known-good Claude Code CLI version"

exit 1

fi

npm install -g "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}"

- Fix GITHUB_OUTPUT redirect: wrap gh api + delimiters in block redirect so output actually reaches the step output variable - Add if: guard for workflow_dispatch where PR number is absent - Network groups: kept as github-only since compiler auto-adds Claude API domains; api-proxy is not a standard network group Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The jq string interpolation with escaped quotes (\"") gets double-escaped by the gh-aw compiler's YAML quoting, producing invalid jq at runtime. Switch to string concatenation (+) which avoids nested quotes entirely. Also simplifies the binary fallback from '"(binary)"' to '""' since binary files with no patch data aren't useful for review. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The PR diff content can contain literal 'EOF' strings (e.g., from heredoc usage in shell scripts), which prematurely closes the GITHUB_OUTPUT delimiter. Use a timestamp-based unique delimiter (GHAW_PR_FILES_<epoch>) that won't collide with diff content. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… delimiter error (#1651) When the PR diff exceeds 8000 bytes, `gh api ... | head -c 8000` causes SIGPIPE (exit 141) on the gh api process. GitHub Actions bash uses `-eo pipefail`, so the pipeline failure aborts the subshell before writing the closing heredoc delimiter to GITHUB_OUTPUT, resulting in: ##[error]Invalid value. Matching delimiter not found 'GHAW_PR_FILES_...' Fix: add `|| true` after `head -c 8000` to swallow the SIGPIPE/pipefail error, ensuring the closing delimiter is always written to GITHUB_OUTPUT. Fixes failing job: https://github.com/github/gh-aw-firewall/actions/runs/23964770751/job/69902358339 Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/c8b8f958-f52e-4390-af7d-f094e20e68da Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

…ch (#1652) Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/0e4673e6-6f23-4e2d-8cc7-ccb922b81091 Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

head -c 8000 can truncate mid-line, leaving no trailing newline. The closing delimiter then appears on the same line as content and GitHub Actions fails to recognize it. Add an explicit echo to guarantee the delimiter is on its own line. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ssions (#1653) Root cause: Claude exceeded max-turns (9 > 8) because mcp__playwright__browser_navigate failed 3 times with EACCES on /tmp/gh-aw/mcp-logs/playwright/*.yml, consuming extra turns. Fixes: 1. Add pre-step to create /tmp/gh-aw/mcp-logs/playwright with chmod 777 before MCP Gateway starts - this ensures the playwright container can write page snapshot YAML files 2. Increase max-turns from 8 to 12 to handle transient playwright retries more robustly Recompile smoke-claude.lock.yml and run post-processing scripts. Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/7c6a8e3c-9a33-488d-a2a7-d54bdaed44a2 Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

github-actions · 2026-04-03T23:25:03Z

Smoke Test Results

✅ GitHub MCP: fix: increase smoke-claude max-turns and fix playwright log dir permissions (#1653), fix: recompile security-guard.lock.yml to fix frontmatter hash mismatch (#1652)
✅ Playwright: GitHub page title verified
✅ File Write: /tmp/gh-aw/agent/smoke-test-claude-23965813992.txt created
✅ Bash: File content verified

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude

github-actions · 2026-04-03T23:25:42Z

🔥 Smoke Test Results

Test	Status
GitHub MCP (latest merged PR: "fix: increase smoke-claude max-turns...")	✅
GitHub.com connectivity (HTTP 200)	✅
File write/read (`smoke-test-copilot-23965814010.txt`)	✅

Overall: PASS

PR by @lpcox, assignees: @lpcox, @Copilot

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-04-03T23:25:57Z

Smoke test matrix (run 23965814007):

fix: increase smoke-claude max-turns and fix playwright log dir permissions, fix: recompile security-guard.lock.yml to fix frontmatter hash mismatch ✅
Safe Inputs GH CLI (safeinputs-gh) ❌
Playwright title check on github.com ❌
Tavily search (GitHub Agentic Workflows Firewall) ❌
File write/read /tmp/gh-aw/agent/smoke-test-codex-23965814007.txt ✅
Bash verification (cat) ✅
Discussion query/comment (github-discussion-query + mystical post) ❌
Build (npm ci && npm run build) ✅
Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-04-03T23:26:41Z

Smoke Test: GitHub Actions Services Connectivity

Service	Check	Result
Redis `host.docker.internal:6379`	`PING` → `PONG`	✅ Pass
PostgreSQL `host.docker.internal:5432`	`pg_isready`	✅ Pass
PostgreSQL `smoketest` DB	`psql SELECT 1` → `1`	✅ Pass

All checks passed.

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-04-03T23:28:06Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #1648 · ◷

lpcox requested a review from Mossaka as a code owner April 3, 2026 21:22

Copilot AI review requested due to automatic review settings April 3, 2026 21:22

Copilot started reviewing on behalf of lpcox April 3, 2026 21:22 View session

Copilot AI reviewed Apr 3, 2026

View reviewed changes